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VOICE MAIL SYSTEM AND METHOD WITH 
SUBSCRIBER SELECTION OF AGENT PERSONALITIES, TELEPHONE USER 
INTERFACE ADDRESS BOOK AND TIME ZONE AWARENESS 

CROSS-REFERENCE TO RELATED APPLICATION 

This application claims priority of U.S. provisional Application No. 60/073,443 filed 
February 2, 1998, the disclosure of which is incorporated fully herein by reference, 

BACKGROUND OF THE INVENTION 

Prior art voice mail systems use series of pre-recorded messages to allow the voice 
mail system to interact with both incoming callers and subscribers to the voice mail system 
(those who have mailboxes on the system). Applications written for the voice mail system 
and the inputs made by incoming callers and/or subscribers determined which pre-recorded 
messages are played and the order in which they are played. These inputs are usually in the 
form of DTMF (Dual Tone Multi-Frequency) tones generated when the caller or subscriber 
presses a button on the telephone keypad. Some of the pre-recorded messages include 
context related information. A common example of such a message is, "You have <number> 
new <message/messages>." The "<number>" portion of the message is replaced by a pre- 
recorded message fragment of the appropriate number for the mailbox being accessed. 
Further, the system chooses whether to use a pre-recorded fragment for "message" or for 
"messages" depending on whether the numeric value of <number> is 1 or another number. 
Prior art voice mail systems typically use one set of prerecorded messages for all subscribers 
and callers. This one set of prerecorded messages also uses a single voice for all of the 
messages in the set. The same person (voice talent) records each of the messages. 

A problem with the prior art voice mail systems is that different people find different 
speech patterns appealing. Different people tend to like and thus better understand different 
voice patterns. Aspects of voice pattems include fast or slow speaking, sing-songy speaking 
or monotonatic, pitch, dialect and the gender of the speaker. Voice pattems also can reflect 
different personalities of the speaker. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of the telecommunications application hardware platform 
according to a preferred embodiment of the invention. 

FIG. 2 is a flow .chart of the process used by subscriber in conjunction with the 
preferred embodiment of selecting an agent for that subscriber. 
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1 DETAILED DESCRIPTION OF THE INVENTION 

A block diagram of the telecommunications application hardware platform 10 used 
with the preferred embodiment is shown in FIG. 1. The typical telecommunications 
applications run on the platform include call answering (including voice, fax and data 

5 processing), multimedia messaging, voice recognition and call management. Despite having 

functionality beyond handling voice calls, for convenience, the telecommunications 
application hardware platform together with its various telecommunications applications will 
be referred to herein as the "voice mail system/' The platform has three main elements, a 
series of media modules 12, a series of application modules 14 and a series of management 

10 modules 16, These three series of modules are interconnected using a dual high speed 
Ethernet backbones, centered around Ethemet Hub A 1 8 and Ethernet Hub B 20. Both 
Ethemet hubs are connected to an Intranet 22, which allows the platform to communicate to 
and receive commands from subscribers through their desktop computers and through the 
Intranet. Alternatively, any network other than traditional Intranets may be employed, 

1 5 including the Internet. A principle kept in mind in the design of the platform is the ability to 
offer extremely high operational availability of the platform and applications running on it. 

The media modules 12 provide media storage, media flow (play/record), media 
transformation (Digital Signal Processing — DSP) resources and the telephony interfaces for 
the platform. The media module is capable of playing and recording multimedia messages 

20 (voice, fax, e-mail) from/to disk as well as supplying tone detection/generation, voice 
recognition, text to speech and fax modem services. 

Each media module uses N+1 power and RAID disk technologies to enhance its 
reliability and availability. Additionally, media modules are clustered to allow failover 
between modules in the event one of them should fail. The operating system for the media 

25 module host CPU is Solaris, 

Application modules are NT servers hosting Microsoft's Windows NT Server 
software. They host the applications and database services for the platform. Commimication 
with media modules is via the media module API and remote procedure calls. Like the media 
modules, the application modules make use of N+1 power and RAID disks and are 

30 themselves clustered for high availability operation. 

The management module is a specialized version of an application module. The 
management module is responsible for hosting the cluster management services and call 
meinager application and is central point for the collection and control of the platform fauh 
management and alarming. To support all of this, the management module contains extra 

35 hardware for connection to media module consoles (not shown), application module 

maintenance ports, and the system monitor. The system monitor provides fault monitoring 
for each of the elements of the platform as well as providing critical, major and minor relay 
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connections to the facility alarm grid. It communicates with each management module across 
an RS232 link. In some configurations, the platform management services and applications 
may reside on the same NT servers. 

The backbone 1 8, 20 is what ties all of the modules together. Each module has dual 
1 00 Base-T Ethernet connections that terminate at two separate Ethernet hubs 1 8, 20. There 
is a third 1 0 Base-T Ethernet connection 22 between the media modules that is used for a 
heartbeat for a failover mechanism. 

A media module consists of a 20 slot VME enclosure, common control, Tl or El 
telephony interface cards, fax service circuit cards (optional), and a storage subsystem. All 
of these components are packaged in a NEBs compliant cabinet enclosure. 

The media module has four main components: media module enclosure power & 
package; common control; telephony/DSP/service circuits; and media storage. 

The preferred embodiments includes a pair of media modules each of which can host 
up to a maximum of 240 telephony and 240 fax ports. However, alternative embodiments 
include additional media modules. 

The media module common control consists of the host CPU, SCSI host interfaces, 
Ethernet controllers and the environmental monitor. 

The media module host CPU is packaged in a three slot, 6U VME bus module. The 
CPU requires a fourth backplane slot when Sbus modules are installed. The Sbus module 
carrier is used int he preferred embodiment. For the preferred embodiment the media module 
host CPU supports: Processor: Dual Hyper SPARC; Memory: 64 to 512 MB ECC DRAM; 
Sbus: Two standard Sbus slots; SCSI: Two fast/narrow SCSI-2 ports; Ethemet: Two 10 Base- 
T Ethemet ports; and Serial I/O: Four RS 232 ports 

The host CPU can support several daughter board processor modules. The preferred 
embodiment v^ll use processor modules with speeds of either 125 MHz, 150 MHz, 166 MHz 
or 200 MHz depending on performance requirements. Single, dual and quad processor 
configurations are possible in altemate embodiments. 

The host CPU can accommodate up to 5 12 MB of error correcting [ECC] memory in 
increments of 64 MB. The ECC provides single bit error correction and multiple bit error 
detection on a per byte basis. The amount of memory populated will depend on performance 
requirements. 

The media module includes a 100 Base-T Ethemet controller that connects to the 100 
Base-T Ethemet hub designated as network B. This network serves as the backup network 
for network A. 

Each media module has an environmental monitor [EM] to monitor and report on the 
system's condition and operating environment. The controller is a single slot VME card 
which the CPU interrogates across the VME bus. 
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The environmental monitor provides: RAID power and fan status for up to 3 RAID 
shelves; Media Modules cabinet temperature; VME enclosure temperature; VME backplane 
voltage monitor (software readable): +5 volts, +12 volts, -12 volts; Cabinet fan status; VME 
enclosure power and fan status; Control of the display panel fault LED; Signaling of media 
module faults to a management module; Remote VME bus reset (to reset the buddy media 
module); Signaling a fault to a management module for the buddy media module; Voltage 
test points for VME backplane voltages; LED indicators for VME backplane voltages; and 16 
LED indicators for media module fault conditions. 

Telephone network and service circuit interface modules provide the telephony, DSP 
and service circuit resources for a media module. 

The modules are made up of a base board and one or more daughter cards combined 
into a single 6U VME64 module. 

The T Wo ice module consists of four Tl spans and the DSP resources to support 
basis voice functionality for 96 channels. The configuration as described below occupies a 
single VME backplane slot. VME base board with an Intel i960 processor, 4 MB of DRAM 
and two Tl spans; Dual span TI daughter board; and Signal processor daughter board with 
six Motorola 66 MHz 56303 DSPs. 

The El A^oice module consists of four El spans and the DSP resources to support 
basis voice functionality for 90 channels. The forth span, allowing 120 channels, will be 
enabled with in a future version of the hardware. The configumtion as described below 
occupies a single VME backplane slot. VME base board with an Intel i960 processor, 4 MB 
of DRAM and two El spans; Dual span Tl daughter board; and Signal processor daughter 
board with six Motorola 66 MHz 56303 DSPs. 

The fax module has the processing resources to provide 24 channels of fax transmit 
and receive functionality in a single VME slot. 

A fax module consists of: VME base board with an Intel i960 processor, 8 MB of 
DRAM; Signal processor daughter board with six Motorola 66 MHz 56303 DSPs. 

Media storage is accomplished using a dual controller RAID system. The RAID 
configuration is 0+1 (mirrored and striped). 

The specifics of the RAID system described here are unique to the Artecon product. 
Alternatively, RAID systems from other manufacturers may be used. SCSI bus termination 
is done extemally to the RAID controllers so that the controllers may be hot swapped without 
interfering with the SCSI bus teraiination. 

An application module consists of an industrial grade PC, common control, and a 
storage subsystem. 
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1 The preferred embodiment supports two pairs of application modules and their storage 

subsystems packaged in a NEBs compliant cabinet enclosure. Each pair of application 
modules shares a single shelf dual controller RAID system. 

Common control of an application module includes the host CPU, a video controller, a 
5 SCSI controller, two Ethernet controllers and an environmental monitor. 

The host CPU is a passive backplane single board computer with an ISA and PCI bus 
interface. It is designed to plug into a PICMG compatible passive backplane that provides 
both ISA and PCI card slots. For the preferred embodiment the application module host CPU 
supports: Pentium Pro processor; custom BIOS; 128 MB ECC DRAM; floppy controller 
10 port; IDE interface; and two RS232 ports. 

The processor in the application module of the preferred embodiment is a Pentixim Pro 
operating at a core speed of 200 MHz and a bus speed of 66.67 MHz. 

The BIOS contains custom extensions to allow remote serial communications with the 
host when the operating system is not loaded. Communication is supported across comm 
15 port 1 . This remote preboot access permits maintenance personnel to interrogate/modify the 
BIOS CMOS settings and to run diagnostics when a system is offline. 

The host CPU can support up to 5 12 MB of error correcting [ECC] memory. The 
preferred embodiment provides 128 MB of ECC memory. The ECC provides single bit error 
correction and multiple bit error detection on a per byte basis. 
20 Application modules are designed to operate without a keyboard or monitor; however, 

a video controller is still required by the Windows NT operating system to allow the system 
to boot. All application modules are equipped with a generic video controller. Depending on 
a particular system, they may be either PCI or ISA cards. The video controller's I/O is not 
routed to the bulkhead, but it is accessible from the rear of the application module chassis. 
25 Each application module has two PCI bus 1 00 MB Ethernet interface cards. The 

Ethernet interface is 1 00 Base-T. 

Each application module has an environmental monitor [EM] to monitor and report on 
the system's condition and operating environment. The controller is a single slot ISA bus 
card which the CPU interrogates across the ISA bus. 
30 The EM has a ISA slave interface that occupies 4 bytes in the ISA I/O address space. 

The environmental monitor provides: RAID power and fan status; application module 
cabinet temperature; application module cabinet fan status; PC enclosure temperature; PC 
enclosure fan status; PC enclosure power status; host processor (Pentium Pro) temperature; 
ISA/PCI backplane voltage monitor: +5, -5 volts, +12 volts, -12 volts; control of the cabinet's 
35 display panel fault LED; control of the PC enclosure's thermal warning LED; signaling of 
application module faults to a management module; CPU reset (to reset itself) Note: a CPU 
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1 reset resets all boards in the AM; remote CPU reset (to reset the buddy application module); 

and signaling a fault to the management module for the buddy application module. 

The application module has a PCI, differential, fast/wide SCSI-2 host controller for 
connection to the RAID system. For compatibility with NT clustering, the controller's SCSI 
5 ID can be changed programmatically. 

Storage is accomplished using a dual controller RAID system. The RAID 
configuration is 0+1 (mirrored and striped). 

Again, SCSI bus termination is done externally to the RAID controllers so that the 
controllers may be hot swapped without interfering with SCSI bus termination. 
10 The management module uses the same components as an application module. In 

addition, it has peripheral storage devices and expansion serial ports to accommodate 
console/maintenance port connections and alarming. 

The preferred embodiment was one pair of management modules and their storage 
sub-system packaged in a NEDs compliant cabinet enclosure. 
1 5 A second PCI SCSI host controller that interfaces to the tape drive is provided in 

management module 2 16b. This controller has a single ended, fast/wide SCSI-2 interface. 

A tape drive and CDROM are equipped in one of the management modules. 
Failover/redundancy is not provided for in these peripheral devices. 

For doing database back ups, a four millimeter SCSI, DAT tape drive is installed in 
20 management module 2 16b. The tape is connected to a SCSI host controller separate from 
the one used for the RAID system. The tape storage capacity is 4 GB. 

For media distribution, installation and upgrade a 12x speed CDROM is installed in 
management module 16a. The CDROM interfaces to the host CPU's on board IDE bus. 
The preferred embodiment uses dual IEEE 802.3 compliant 1 00 MB Ethemet 
25 networks to connect all of the intemal modules. Both networks are built around 12 port 100 
MB repeater hubs that connect to each of the networked modules. 

Each hub has a single power supply. The hubs are connected to separate input power 
feeds to allow maintenance to be performed on one of them without disturbing the other. 

Two ports from each hub are routed to the management module's I/O bulkhead. These 
30 provide the external network access to the cluster. 

Each medial module can support a maximum of 240 telephony ports. The ports can be 
either Tl or El , but both may not be mixed within the same backplane. 

For a Tl system a maximum of three 4 span line cards may be installed in each media 
module. In this configuration only two of the four spans are usable on the third line card. 
35 For an El system a maximum of three 3 span line cards may be installed in each media 

module. In this configuration only two of the three spans are usable on the third line card. 
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1 The preferred embodiment supports a 24 port fax card. Each media module can 

accommodate a maximum of 10 fax cards for a total of 240 fax ports. 

The minimum number of disks allowed in any RAID configuration is 3. In the media 
module the maximum number of RAID disks allowed is 15 with 14 available for storage and 

5 1 reserved for a global spare. In the application and management modules the maximum 

number of RAID disks allowed is 7 with 6 available for storage and 1 reserved for a global 
spare. In each RAID system two slots are reserved for building new RAID sets when 
performing disk upgrades. 

The platform is designed for modular hardware serviceability. The foUov^ng 

10 components support live insertion/removal and may be added or replaced with no impact on 
system operation: media module (including RAID disk drives and controllers; VME & RAID 
power supplies & fans; telephone network interface modules; fax service circuits; and 
CDROM drive); and application module/management module (including RAID disk drives 
and controllers; AM/MGM & RAID power supplies & fans; and AB sv^tch power). 

1 5 If the need arises to replace a component that does not support hot swap, redundancy 

allows failing over to another resource during the maintenance period. 

The platform provides the hardware capability to perform software upgrades without 
shutting the entire platform dovm. Upgrades are perfomied incrementally on each module 
until all modules are at the new software level. There may be some degradation in the level 

20 of service during the time a module is being upgraded. 

For upgrade purposes, a single media module may be removed from service, upgraded 
and returned to service with the only result being reduced port capacity during the down time 
interval. 

An individual media module must be removed from service to upgrade any of the 
25 common control components. 

Network interface modules are hot-plug and may be added or removed from a media 
module without service interruption. 

Fax cards are hot-plug and may be added or removed from a media module with 
service interruption. 

30 Increased storage upgrades are expected to occur with each advancement made in disk 

drive storage capacities. The platform architecture provides a transparent upgrade path that 
requires no down time to the media module being upgraded. Requirements for the upgrade 
axe as follows: two open slots in the RAID subsystem; and all disk drives in a logical unit 
must be upgraded at the same time. 

35 The upgrade capabilities for the application and management modules are identical. 

The application modules are configured in redundant pairs. One application module 
may be removed from service and upgraded without interrupting service to the cluster. 
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1 Application and management modules must be removed from service to service any of 

the common control components. 

In the preferred embodiment there is no present need to upgrade the Ethemet networks 
exists, however, the platform Architecture cillows for network upgrades to accommodate 
5 expansion in future releases. As the cluster grows, the network structure can be migrated to 

higher speed Ethemet, ATM of other network technologies. 

In the preferred embodiment, the application modules include the applications that 
supply the flmctionality of voice mail system that is apparent to a user of the voice mail 
system. As stated above, the applications typically include the functionality of call 
10 answering (including voice, fax and data processing), multimedia messaging, voice 

recognition and call management, it can include any telecommunications functions. In the 
preferred embodiment, the application includes several sets of pre-recorded messages. 

All of the messages of a particular set of pre-recorded messages are recorded by the 
same voice talent. However, different sets of pre-recorded messages may be recorded by 
1 5 different voice talents. Additionally, each set of pre-recorded messages is recorded by the 
voice talent using the same basic speed, dialect, pitch and personality. Of these different 
voice qualities used in recording a set of messages, personality is of importance. The same 
voice talent can record messages using any of a variety of distinct personalities and emotional 
qualities, such as happy, serious, verbose and terse. When multiple voice talents are also 
20 used, the number and variety of different voice/personality combinations is considerable. 

The preferred embodiment then allows the user to select one from the various sets of 
pre-recorded messages stored in the application module. Once a set of pre-recorded 
messages is chosen, that set represents the subscriber's chosen "agent" for interacting with 
the voice mail system. The preferred embodiment employs a brief interview process shov^ 
25 in FIG. 2 to allow the user to select the agent with the personality preferred by the user. 

When a subscriber first uses the voice mail system, or decides to change their agent, the 
selection process begins at entry point 30. In block 32, the system begins waiting for a voice 
selection response from the subscriber. At the same time, a Sample Introduction, which 
introduces the agents to the subscriber, is played. In one embodiment the Sample 
30 Introduction follows the following script. In the following script. Agent #1 is the default 
agent in the case of a new subscriber, or the currently selected agent for an existing 
subscriber who is changing their agent. 

Agent # 1 : You may select among <number of voices available (n)> assistants. This 
is what we sound like . . . 
35 Agent #1 : It's either my voice, the one you've heard before . . . 

Agent #2; ... or it's me. You can hear my voice ... 
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1 Agent #n: ... or you and your callers can hear my voice. 

Agent # 1 : For my voice, press 1 . 
Agent #2: For my voice, press 2. 

5 Agent #n; For my voice, press <n>. 

It is noted that the phrase used to introduce each new voice (the "personality signature 
phrase"), such as Agent #2's . . or it's me. You can hear my voice . . usually contains 
slightly different words to make up the introduction. The words used are chosen to be 
consistent with the agent's personality. 

10 If the user responds to the Sample Introduction by pressing a key on their telephone 

between 1 and n, indicating a valid choice of an agent, the branch to block 34 is taken. In 
block 34, a message, spoken by the just-chosen agent, states, "OK, Til be your assistant from 
now on." Of course, the words used by the chosen agent to communicate the choice of that 
agent may be different for each agent, reflecting that agent's personality. After block 34, the 

15 agent selection process exits at block 38. 

If the user does not respond to the Sample Introduction or responds with an 
inappropriate key (not a number between 1 and n), the path from block 32 to block 36 is 
taken. In block 36, a message, spoken by the prior agent (Agent #1 in the case of a new 
subscriber), states, "OK, I'll continue to be your assistant." Again, the words used by the 

20 chosen agent to communicate the retention of the prior agent may be different for each agent, 
reflecting that agent's personality. After block 36, the agent selection process exits at block 
38. 

With respect to the pre-recorded messages needed to implement the agent selection 
process, the voice for each agent must record all of the numbers from 1 to n £uid each of the 
25 following phrases; 

You may select among <number of voices available (n)> assistants. This is what we 
sound like ... 

It's either my voice, the one you've heard before . . .; 

["middle of list" intonation] <personality signature>; 
30 ["end of list" intonation] <personality signature>; 

For my voice, press . . .; 

OK, I'll be your assistant from now on; and 

OK, I'll continue to be your assistant. 
Each of these phrases is needed for each agent because the current choice of agent, which 
35 could be any of the agents for a particular subscriber, determines which agent's recordings 
are use for each of the phrases. Again, the specific wording used by a particular agent to 
convey the message of each of the phrases can vary with the personality of the agent. 
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1 It has been found that even the short exposure to a voice and personality that is given 

in the agent selection process shown in FIG. 2 supplies subscribers with a great amount of 
information from which to choose a favorite agent. In particular, most subscribers can 
immediate pick out their favorite voice/personality from one listening to the short personality 

5 signatures provided to them. Altematively, though, longer exposure to the different agents 

can be provided. In another alternative, the system can conduct an interview process to try to 
determine for the subscriber which agent is most likely to appeal to them. 

As voice mail system subscribers use a particular voice mail system more often than 
outside callers, the choosing of a personal agent is more important for subscribers than 

1 0 callers. However, using ANI information contained in the data the voice mail system 

receives with each incoming call that identifies the calling stations phone number, the voice 
mail system can automatically use a selected agent for all calls received from a particular 
phone number. Alternatively, Caller ID information or any other method of determining the 
calling party's number can be used in place of or in addition to ANI information. In this way, 

15 the time involved in selecting an agent by a caller can be worthwhile for callers who call the 
particular voice mail system often from the same phone number. Altematively, known voice 
recognition technology can be used to identify incoming callers by analyzing the spoken 
response to a system generated prompt, such as, "Hello, you've reached the office of Joe 
Smith, whom may I say is calling?" The response is then processed by the voice recognition 

20 software and if the caller is recogni2^d, the caller's choice of agent is then used. 

In a combination of the ANI and voice recognition methods of caller identification, a 
database is provided which stores correlations between callers identified by the voice 
recognition software and the number from which they have called, according to the ANI 
information. Thus, when a call comes into the system, the database is first checked to see if 

25 there are any entries which corresponding to the ANI infomiation for the incoming call. If 
there is only one entry for the calling number, the system answers the call, "Hello, is this Joe 
Smith?" using the agent that Joe Smith has chosen before. Then, analyzing the response to 
this prompt, the voice recognition software can verify that the caller's voice matches the 
systems records for Joe Smith's voice. If the voice does not match, or says "No", the system 

30 first checks to see of the voice of the response matches any other records in the database. If 
so, the system can ask if the caller is the person in the database for the matching voice to 
verify their identity. If the system does not find a match for the voice in the database, or if 
the caller does not verify their identity as selected from the database, the system proceeds to 
add a new entry for the caller, with both the ANI information and voice pattern information. 

35 Using the same methods, a subscriber can be identified when they call into the system 

from an outside telephone. In this instance, the subscriber himself or herself has an entry in 
the database. This database includes the subscriber's various telephone numbers, such as 
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1 home, office and mobile numbers. Thus, if the system receives a call with ANI information 

matching any of the subscriber's various telephone numbers, there is a high likelihood that 
the caller is the subscriber. When an outside caller is identified by the system as a subscriber, 
that subscriber's agent is used to handle to call. 

5 The identification methods also allow a subscriber to select an agent for anyone in 

their address book. This is of particular usefulness when the subscriber knows that a person 
in the database is more comfortable with a language different than the language used by the 
system's default agent, hi an example of this situation where a particular caller prefers to 
speak French, the subscriber can have the voice mail system always use a French language 

1 0 agent once it has identified that caller. 

Additionally, the voice mail system can be set up to allow outside callers who are 
frequent users of the voice mail system to choose an agent that is preferable to them. Again, 
as with the caller who prefers a different language, once the caller is identified by the voice 
mail system, the database is accessed and the caller's chosen agent is used for the duration of 

15 the call, hi this case, though, no subscriber involvement is necessary in the selection of the 
preferred agent for that caller. 

In an alternative embodiment, using known weaving technology (voice patterns of 
someone reading a sentence or paragraph are taken and the words/phonemes in the sample 
are used to make new sentences) the system can create an agent based upon the user's own 

20 voice and speech patterns. Thus, a user reads a paragraph (or however much sample is 

needed) and the recording of that reading is used to automatically create the pre-recorded 
messages for that user's agent. 

The database that stores information about callers based on ANI information, voice 
recognition or a combination of these techniques allows further fimctionality to the voice 

25 mail system apart fi-om the selection of a preferred agent personality. This additional 

functionality centers around the concept of a personal assistant, enabled by expanding the 
incoming caller database into a telephone user interface (TUI) address book. The address 
book contains phone numbers, fax numbers, other numbers, e-mail addresses and other 
information that will facilitate intelligent handling of calls to and fi-om the people in the 

30 address book. 

Each entry in the address book contains an entry number, a caller-spoken name 
utterance, a subscriber-spoken utterance of the entry's name, the entry*s phone number, and 
the entry's fax number (optional). As caller messages are left for the subscriber with the 
voice mail system, the caller-spoken names, ANI and keyed-in phone numbers are saved with 

35 the message (if requested by the greeting option selected and given by the caller). When the 
subscriber has listened to a message for which at least some phone number information is 
available, one option that becomes available is to add this caller to the address book. If this 
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1 option is selected, the voice mail system will prompt the subscriber to say the spoken name of 

the caller, verify the phone number to be stored, and enter the fax number if available and not 
already entered by the caller. 

Names may also be added manually by selecting the appropriate option under a 

5 "Setup/Address Book" menu of the voice mail application. After selecting the slot into which 

the entry will be placed, the subscriber-spoken utterance of the entry's name, the phone 
number and the fax number are entered manually. 

Each slot in the address book is numbered, starting with one. When a person is 
assigned to a slot, they stay there with that number until manually deleted or replaced via the 

10 address book maintenance options (Add, Change, Delete, Review One, Review All) in the 
"Setup/ Address Book" menu. Deletions do not reorder the list, and additions are placed in 
the slot number specified by the subscriber (the first available slot is suggested by the 
system). Any information in a slot may be replaced (Change command). 

One fiinction available to the system with TUI address book is calling people from the 

1 5 information in the address book. To call a person in the address book the subscriber presses a 
number associated the person. If there are 1 0 or more entries, the selection of the entry to 
dial is considered complete when the key is pressed or a timeout occiurs. If the number of 
entries is less than 10, the attempted outdial occurs immediately upon receipt of the single- 
digit keypress, without waiting for a timeout. The phone number fi-om the address book is 

20 then used to dial the desired person. 

Alternatively, the phone number for a person in the address book can be dialed when 
the subscriber speaks the person's name to the voice mail system, using voice recognition. In 
one embodiment, the voice recognition relies on the subscriber having spoken the person's 
name to the voice mail system when the person's address book record was created. In 

25 another embodiment, a voice recognition name template is created fi-om the text 

representation of the person's name and stored as an entry in the address book. The person is 
then dialed by a voice command matching the voice recognition name template. 

Another feature is automatically adding an entry to the subscriber's address book from 
the information stored in an incoming voice message to the subscriber. Particularly if the 

30 voice mail application prompts the caller to say their phone number at a particular point in 
the call, the phone number is converted into digits which are automaticdly stored in the 
caller's entry in the address book should the subscriber add the person to the address book. 
The address book also allows the automatic addition of a person to the subscriber's address 
book from the information stored in an incoming e-mail message to the subscriber. 

35 The system stores extensive information about the subscriber's use of a particular entry 

in the Address Book as part of the entry. Thus, for example, when there is a question by the 
voice recognition software whether the subscriber said "John Smith" or "Joe Smith," the 
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system can make an intelligent guess that the subscriber intended to dial Joe Smith because 
that subscriber calls Joe Smith once a week on average and that subscriber hasn't called John 
Smith in over six months. 

The system also stores information about the number and frequency of calls from a 
particular entry in the address book as part of the entry, as well as storing information about 
how the subscriber treats this caller and/or messages from this caller. With this information, 
the system can predict how the subscriber will want to treat a particular call/message. For 
example, if the subscriber always immediately retums calls from their mother, the voice mail 
system presents the message from the subscriber's mother and automatically asks if the 
subscriber would like the system to place a return call immediately. Altematively, if a 
subscriber routinely sends calls from a particular caller to be answered by the voice 
messaging fimction, the system will present the call with a statement such as, "Mr. X is 
calling, I will route the call to voice mail unless you press the star key . . The same 
information can be used by the voice mail system to make an internal decision to promote an 
address book entry to a VIP list based on subscriber's treatment of calls and messages from 
that entry. 

In an alternative embodiment, a visual graphic based computer interface is used to 
communicate between the voice mail system and the subscribers in all of the functions 
described above instead of using voice based menus used over the subscriber's phone as 
described above. 

Another feature of the preferred embodiment is automatically adjusting which pre- 
recorded messages are played at different points in user interaction based upon the user's 
competence in interacting with the system, as perceived by the voice mail system. In almost 
any particular point in the interaction with a voice mail system, while there is one particular 
piece of information that needs to be communicated to the user, the manner of 
communicating this information can range greatly along the spectrum from terse to verbose. 
Generally, experienced users prefer more terse messages while inexperienced users 
appreciate more verbose messages. In a preferred embodiment, the system monitors various 
parameters such as the frequency at which the user reaches a particular point in the system 
where a message is played, errors made by the user (determined by hangups followed by 
immediate return calls, backups in a menu hierarchy, etc.), multiple long pauses without user 
input at the same point in the system on consecutive calls (as opposed to singular pauses 
which may be the result of the user being distracted/interrupted while using the system), and 
how quickly a user interrupts a message with a selection (e.g., DTMF tone). 

A preferred embodiment allows the use of these detection criteria to be used both for 
changing the message played at a local point in the system or for making global changes in 
the level of message played to a particular user. This process is called flexing the prompt 
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1 levels and the specific way the various parameter are programmed to affect the flexing of 

prompt levels is called the flex behavior of the system. Of course, the user is allowed to 
override the system and manually select the terseness of messages either locally or globally. 
Last, as with the selectable agent personalities, while mostly applicable to internal users of 
5 the system, the invention may also be applied to incoming callers, identified by the ANI 

information received with the call or any other manner of identifying the caller, including 
voice recognition as discussed above. 

The system also allows the fine-tuning of the flex behavior by individually setting all 
of these "flex-triggers" on a state-by-state basis, allowing the application designer very 
1 0 precise control over flex behavior throughout the application. For example, items on the main 
menu, used more frequently, might flex after 5 correct uses, whereas the selection of a 
greeting style (which includes a long preamble before the menu at the beginner level) might 
flex after only two uses. 

Further, this would also allow an entire set of flex settings to be associated with a 
15 "language," allov^ng the designer to adjust flex behavior to be appropriate for various 
languages/personalities provided with the application. 

In a preferred embodiment, there are three levels of prompting and four possible 
settings of a subscribers prompt level: 

Flex - this setting automatically adjusts prompts among the three available levels to 
20 accommodate the subscriber's experience level at a number of pre-defined states in the 

user interface, adjusting the level up (or down) as appropriate. 
Training - these are explanatory prompts that provide all of the information a 
subscriber might require to understand what the next appropriate response on their part 
needs to be. 

25 Standard - these are concise prompts, asking for the subscriber's response directly, 

with little or no explanation; they assume the subscriber has progressed beyond the 
"training" stage. 

Advanced - these are tightly edited prompts, using only sounds or very short verbiage 

to prompt for the desired response. 
30 When the Flex level is selected, at pre-defined states in the user interface, the preferred 

embodiment adjusts its prompts among the three available levels to match the subscriber's 
level of experience with a particular activity. In any state within which prompt level is not 
tracked, prompt level is inherited fi-om its parent state. 

In order to adjust prompt levels appropriately, the system tracks the subscriber's 
35 experience at each of the predefined states within the call flow, adjusting in either direction 
based on both usage and the length of time since the last use of the function. For example, if 
the subscriber successfully completes a given action (based on valid keypresses and/or error 
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1 tones received) a certain number of times, the prompts are abbreviated by one level. If, on the 

other hand, they fail a certain number of times, or if they have not attempted this particular 
action for more than "n" calendar days, the prompts are lengthened by one level. 

The flex level is also adjusted based upon a combination of the total number of times a 

5 particular menu item has been used by a subscriber and the number of uses in a recent time 

period by the subscriber. Thus a subscriber who has used a menu item many times in the 
past, but has not used it in the last three months may be given a lower flex level than a 
subscriber who has not use the menu item nearly as many times total, but has used the menu 
item ten times in the last week. On the other hand, the subscriber who has used the menu 

1 0 item many times in the past, but has not used it in the last three months should probably be 
placed at more advanced flex level than a subscriber who has used the menu item only two 
times, but both uses were in the last month. 

Whenever the subscriber responds to a particular prompt with silence, the Training 
level's version of that prompt is played next, providing more information about the response 

15 expected (no permanent adjustments are made to prompt level; this is isolated behavior on a 
prompt-by-prompt basis). 

The subscriber has the option to lock all interaction at a certain level, regardless of 
experience. (In areas where lack of instruction could result in loss of information or other 
undesirable result, verbose prompts are given, even at the Advanced level). 

20 In another embodiment of the invention the voice mail system is implemented with 

time zone awareness. Some prior systems in other fields have some time zone related 
features. For example, Windows 95 (Windows 95 is a registered trademark of Microsoft 
Corporation) requests the user to select their time zone when Windows 95 is installed. 
Thereafter, the computer is aware of its time zone, including when to change to daylight 

25 saving time and back. In prior voice mail systems, messaging systems are unaware of any 

time zones outside of their own. If, for example, the voice mail system places a "time stamp" 
on an incoming message for a user, the time stamp relates to the time zone in which the voice 
mail system is located. The same is true for outgoing calls; a call to a given destination may 
not be in the same time zone as the caller, but the prior systems are unable to make the caller 

30 aware of any time zone differences. 

In this alternative embodiment, the "time stamp" used for incoming messages is not 
necessarily tied directly to the time zone of the voice mail system, but rather is tied to a flxed 
time zone, such as Greenwich Mean Time (GMT). At the same time, the central clock for the 
voice mail system is set to the fixed time zone, such as GMT, The system is then 

35 programmed with the time zone that it presently is in and the difference in time between the 
progranuned time zone and GMT is used to calculate the local time, for any purpose the local 
time is needed by the voice mail system. For messages, with the time stamp recorded in this 

-15- 



SDOCID: <WO_9939499A2J_> 



wo 99/39499 



PCT/US99/02238 



1 manner, the system then calculates the time of message deposit by a) the time that has passed 

since the message was deposited (e.g., "two and a half hours ago"), b) the time at the message 
depositor's location (e.g., "2:00 p.m. local time from caller's location"), c) the time at the 
user's/subscriber's location (e.g., "3:00 a.m., your time"), and d) the time at the 

5 user's/subscriber's nomial or "home" location. 

In the above examples, the time at the message depositor's location and the time at the 
user's/subscriber's location are calculated by first obtaining the phone number the caller or 
the user/subscriber is calling from. As discussed above, this can be done using the ANI or 
Caller ID information in the incoming call. Once the incoming phone number is determined, 

10 a database is consulted, which includes international telephone country codes, area codes and 
even telephone prefixes and correlates these to specific time zones. In the United States, the 
database must go to the prefix level of detail because there are area codes that span more than 
one time ^one. For the user's/subscriber's normal or "home" location, the time zone for the 
user's/subscriber's normal location is entered into the system by any of a variety of ways, 

15 including the voice mail system asking the user/subscriber to supply this information at the 
time their mailbox/account is set up and the information being input or programmed into the 
voice mail system by a system operator. 

Additional functionality of this embodiment include alerting users of the system of the 
local time of party whose phone number the user has just dialed. The same database 

20 described above that correlates country codes, area codes and prefixes to time zones is used 
by the system to determine the time zone for a dialed phone number. The system is 
programmed, on a user by user basis if desired, to inform the user of the called party's time 
zone always, when the called party is in a different time zone that the caller or when the time 
at the called party's location is within a certain range (e.g., 10:00 p.m. to 7:00 a.m.). Utilized 

25 in combination with the TUI address book, when the system informs the caller of the called 
party's current time can also be programmed individually according to an individual in the 
TUI address book or according to the called phone number itself. For example, the user's 
mother may go to bed early and get up early, so an alert between 9:00 p.m. and 6:00 a.m. may 
be appropriate. The caller's brother, on the other hand, may be a night owl and an alert 

30 between 2:00 a.m. and noon may be appropriate for him. Similarly, using the same database, 
the time of day for the called party in the case of outgoing calls or for the calling party in the 
case of incoming calls is displayed on the display or screen of the user of the voice mail 
system. 

Another aspect of this embodiment, is that the priority for incoming voice mail 
35 messages may be adjusted based on the time of day of the message at the caller's location. 
Thus, a call placed at 2:00 a.m. from a caller may have its priority level raised. Again, the 
particular circimistances or time ranges in which a message's priority may be changed based 
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1 on the time of day at the caller's location is adjustable at the individual or phone number 

level in the TUI address book. 

The time zone/telephone number database can be enhanced by adding city and country 
information. Thus, in the case of a user calling in for messages from the road, instead of the 

5 voice mail system simply saying, "The message was left at 3:00 a.m., your time." the system 

would say, "The message was left at 3:00 a.m., your time, in Tokyo." In this manner, the 
user would be provided reassurance that the voice mail system had correctly determined the 
user's location and that the time information being received is accurate. 

In another aspect of this embodiment, the time zone/telephone database is utilized in a 

1 0 auto-attendant system. For example, in a message that informs the caller of the operating 
hours of a customer service phone line, a caller in California may hear a message, "The 
customer service line is open from 6:00 a.m, to 4:00 p.m.. Pacific Standard Time." while a 
caller in New York calling the same customer service phone line would hear the message, 
"The customer service line is open from 9:00 a.m. to 7:00 p.m., Eastem Standard Time." 

15 
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1 CLAIMS: 

1 . A telephone system comprising: 

means for detecting the phone number of the phone from which an incoming call is 
being made; 

5 a database relating portions of phone numbers with time zones; 

a clock; 

means for determmmg the difference in time zones between a detected phone number 
of a party placing an incoming call and the time zone of the party receiving the incoming call. 

10 2. The system of clahn 1 comprising means for inclusion of an indication of the 
determined difference in time zones in a message related to the incoming call. 

3. The system of claim 1 comprising means for communication an indication of the 
determined difference in time zones to the party placing the incoming call. 

15 

4. The system of claim 1 comprising means for communication an indication of the 
determined difference in time zones to the party receiving the incoming call. 

5. The system of claim 1 wherein the database also relates portions of phone numbers 
20 with cities and countries. 

6. The system of claim 1 fiirther comprising means for generating a message to the party 
placing the incoming call that at least partially depends upon the determined difference in 
time zones. 
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